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ABSTRACT 

We introduce an interactive online resource for use by students and college instructors in introductory as¬ 
tronomy courses. The General Education Astronomy Source (GEAS) online tutor guides students developing 
mastery of core astronomical concepts and mathematical applications of general astronomy material. It con¬ 
tains over 12,000 questions, with linked hints and solutions. Students who master the material quickly can 
advance through the topics, while under-prepared or hesitant students can focus on questions on a certain topic 
for as long as needed, with minimal repetition. Students receive individual accounts for study and course in¬ 
structors are provided with overview tracking information, by time and by topic, for entire cohorts of students. 
Diagnostic tools support self-evaluation and close collaboration between instructor and student, even for dis¬ 
tance learners. An initial usage study shows clear trends in performance which increase with study time, and 
indicates that distance learners using these materials perform as well as or better than a comparison cohort of 
on-campus astronomy students. We are actively seeking new collaborators to use this resource in astronomy 
courses and other educational venues. 

Subject headings: astronomy education research; astronomy education resources; college-level astronomy; 
distance education; STEM coursework 


1. INTRODUCTION 

Students in introductory college science courses often ar¬ 
rive academically underprepared. In 2011, just 43% of high 
school graduates met the American College Testing Program 
(ACT, 2014) benchmark for mathematics, and only 37% met 
the benchmark for science. High school preparation in math¬ 
ematics is a significant predictor of success in introductory 
college science coursework (Sadler & Tai, 2007), and it is 
sobering to realize that fewer than half of high school grad¬ 
uates currently meet the benchmark. Even more worrisome, 
of students interested in careers in education, only 37% met 
the ACT math benchmark in 2011 (30% for science) (ACT, 
2014). As 40% of students taking general education sci¬ 
ence intend to become teachers (Lawrenz, Huffman, & Ap- 
peldoorn, 2005), improvements can be leveraged to aid K-12 
level educational efforts. 

The last few years have seen tremendous growth in In¬ 
ternet access, with 95% of American college students hav¬ 
ing home broadband access and 100% having access through 
some source (Pew Research Center Internet & American Life 
Project, 2011). Online programs for homework, testing, 
and assessment in varied subject areas have been developing 
steadily in response (Bennett, 2001; Koedinger et al., 2013), 
particularly within the sciences (cf., Bonham, Deardorff, & 
Beichner, 2003; Donovan & Nakhleh, 2007; Kumar, 2005). 
Opportunities for intensive, private, interactive study of key 
concepts can help students to develop skills without embar¬ 
rassment, to gain knowledge, and to succeed. Online sys¬ 
tems can provide repeated exposure to concepts, integration of 
mathematics with scientific material, and instantaneous feed¬ 
back (Nguyen Hsieh, & Allen, 2006), as well as increasing 
access for students at a distance and/or asynchronously. 

There are several rich content resources for stand-alone 
astronomy questions, including Green’s hundreds of Con- 
cepTests (Green, 2002) and the thousand questions col- 
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lected by the Collaboration of Astronomy Teaching Scholars 
(CATS; Wallace et al. 2012). Online tutors for astronomy 
are offered through several publishing houses, where these 
resources are typically bundled with textbooks, and students 
pay a sizable fee for access for a single semester (Barmby, 
2010). Our self-review library, an interactive online tutor that 
works with students to develop mastery of core astronomical 
concepts and mathematical applications, can be distinguished 
both in terms of its scope (containing 12,000-f questions, pro¬ 
viding detailed student diagnostics and individual, adaptive 
feedback), accessibility (runs in any desktop or mobile web 
browser) and affordability (free). Pilot usage studies have 
been conducted at New Mexico and California colleges, and 
our resources are now being offered nationwide. 

2. STRUCTURE OF THE GEAS SELF-REVIEW LIBRARY 

The General Education Astronomy Source (GEAS) self- 
eview library is an intelligent online tutor built around 26 
modules, each based on a 75-minute lecture. The modules 
follow a common pattern for a semester-long course in gen¬ 
eral astronomy. Each module is presented as a series of web 
pages (200 in total), augmented with slide-by-slide audio lec¬ 
tures and reproductions of white board diagrams from con¬ 
ventional classroom lectures. A printed textbook is no longer 
required in our classroom. We have found (as have others, 
cf. National Academy of Sciences, 2007; Stavrianeas, Stew¬ 
art, & Harmer, 2008) that the majority of the students perform 
well with alternative, no-cost, resources. A set of 440 Think- 
Pair-Share questions (Lyman, 1981) have also been created, 
developed for the modules in the form popularized via Na¬ 
tional Aeronautics and Space Administration (NASA) Center 
for Astronomy Education (CAE) workshops (cf. Forestell, et 
al. 2008). 

Material is presented at a level accessible for non-science 
majors, with a balance between qualitative and scaling argu¬ 
ments and numerically motivated ideas. The basic mathemat¬ 
ics needed for the material (algebra, roots, exponents, and sci- 
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entific notation) is presented early on and reinforced through¬ 
out the sequence. The art of visualization is presented as an 
important problem-solving strategy and emphasized for top¬ 
ics for which it is most useful (e.g., understanding the phases 
of the Moon, eclipses, and orbital mechanics). 

2.1. Pattern of Usage each Week 

Figure 1 shows the distribution of self-review library us¬ 
age, identifying study patterns each week for a cohort of dis¬ 
tance learners. While most studying took place between noon 
and midnight, there is a tail of activity extending out past 3 
a.m., and another picking up at 6 a.m. (e.g., a night watch¬ 
man studied primarily in twenty-minute blocks spread out be¬ 
tween midnight and eight am, in between making his rounds.) 
Individuals work in blocks of time of average length 22 min¬ 
utes; the broad features shown thus represent the overlapping 
efforts of multiple students. Efforts are concentrated before 
weekly deadlines. 

Students were divided into two groups based on their rat¬ 
ings for the tutor after a semester of usage. They responded 
positively overall to the tool, with 9% finding it acceptable, 
41% classifying it as a better than average study tool and an¬ 
other 50% rating it as one of the best study tools they had ever 
seen. Figure 1 contrasts the study patterns of these last two 
groups. We find that the most enthusiastic users completed 
16% more problems while studying, and tended to study sig¬ 
nificantly more in the afternoons leading up to weekly dead¬ 
lines. 

2.2. Topical (Lecture) Modules 

Table 1 lists the 26 topic modules and the number of self¬ 
review questions of various types for each module. The se¬ 
quence begins with a short set of three modules (1-3) that 
present an overview of the astronomical sequence and review 
the math skills necessary for the entire sequence. Questions 
within these modules are designed to be more straightforward 
and are written at a simpler level than the average, to give stu¬ 
dents a chance to become familiar with and to adjust to using 
the interface. The next two modules (4-5) introduce the con¬ 
cept of developing a model (through the scientific method) 
that enables us to predict the future behavior of physical ob¬ 
jects. Modules 6-8 and 10-13 focus on the history and behav¬ 
ior of objects within the solar system. There is a continued 
emphasis on visualization when appropriate, as well as pre¬ 
sentation of new astronomical knowledge. Module 9 focuses 
on the scientific method. It also contains a review of addi¬ 
tional mathematical applications, including linear fits and the 
concepts of a histogram, a normal distribution, a mean value, 
and a standard deviation, useful for laboratory experiments 
(Vogt, Cook, & Muise, 2013). These first 13 modules com¬ 
prise the first half of the sequence and fall loosely under the 
umbrella of “solar system” material. 

The second set of modules covers stellar and extragalac- 
tic astronomy topics. Modules 14-15 introduce the con¬ 
cepts of light as a wave with an associated frequency and 
energy and the modern model of the atom. In modules 16- 
18 the absorption and emission of photons within the atom 
is connected to macroscopic observables such as stellar spec¬ 
tra for stars of various masses and temperatures. Module 19 
delves into Doppler shifts and binary star systems, and mod¬ 
ule 20 combines the ideas developed in 16-19 to motivate the 
Hertzsprung-Russell Diagram of stellar evolution. Modules 
21-23 describe the life cycles of stars of various masses. The 


final three modules (24-26) introduce ideas about galaxies 
and the cosmological structure and expansion of the Universe. 

2.3. Self-Review Questions 

Table 1 lists the number of self-review questions for each 
module, 12,406 in total, divided into several groups of inter¬ 
est. The variations in the mean values of student attempts by 
lecture refiect variations in difficulty between topics. The two 
modules (4-5) with the lowest means develop the ability to vi¬ 
sualize three-dimensional movements of solar system objects 
such as the Earth, Moon, and Sun system, and the next lowest 
(Module 18) introduces the mathematically challenging con¬ 
cept of radioactive decay rates. 

There are groups of similar questions within each module, 
all derived from one template/archetype problem. This is par¬ 
ticularly true for the numerical and spatial-visualization prob¬ 
lems which students find most challenging. The column la¬ 
beled “AT” defines the number of archetype questions that are 
independent from one another (having different content, and 
not containing large clues to each other’s answers). Ques¬ 
tions labeled as “MC” are multiple-choice questions - a cat¬ 
egory which contains both traditional multiple choice ques¬ 
tions, ranking tests, and figure analysis exercises - while those 
labeled “NV” are numerical value questions. For these ques¬ 
tions, the student enters a number rather than selecting one of 
several answers, the value of which must lie within a certain 
range to be considered correct. The final column lists the total 
number of questions within each module. There are on av¬ 
erage 55 archetypes per module, and 477 questions based on 
these archetypes split fairly evenly between the MC and NV 
modes. Each individual question contains three components: 
the question proper, a tailored hint that provides guidance but 
does not reveal the answer, and a fully worked solution to the 
problem. 

Questions are designed for each module to give experience 
in a variety of problem-solving modes, and with an emphasis 
on quantification and on extrapolation, drawing on physical 
evidence and theory to deduce logical properties and patterns 
found in the physical universe. Question types include the 
following: 

(A) Extrapolation: How can our knowledge of the biodiver¬ 
sity found on Earth enable us to estimate the probability 
of finding life in the oceans of Europa? [Modules 12 
and 13] 

(B) Scaling: If different galaxy components have different 
spectral energy distributions (emitted flux as a function 
of wavelength), how should optical and infrared images 
of the bulge or disk of a spiral galaxy like the Milky 
Way differ from each other? [Module 26] 

(C) Visualization: Given a physical model of the Sun- 
Earth-Moon system, what is the observed phase of the 
Moon at a certain position in the sky at a certain time 
of day? [Module 4] 

(D) Figure analysis: What is the frequency of a displayed 
light wave? [Module 14] 

(E) Computation: If the Sun burns its reservoir of hydrogen 
at a certain rate, how long can it exist in the hydrogen¬ 
burning phase? [Module 17] 

(F) Algebra: If the Hertzsprung-Russell diagram shows us 
the specific relationship between the luminosity, tem¬ 
perature, and radius of a star, how can one calculate one 
quantity from the other two variables? [Module 17] 

Students interact with the self-review library by requesting 
a five-question quiz. Each question is presented with two aids. 
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Fig. 1.— The weekly pattern of self-review library usage for distance learning students over a semester. The red dotted line shows the distribution of 7,160 
questions completed by 22 students who ranked the library as a good study tool, better than average, and the blue solid line shows the distribution of 10,210 
questions completed by 27 students who ranked it as one of the best tools ever (5 students who ranked it as typical are not shown). Histograms are scaled so 
that equivalent amounts of work per student within each group would produce histograms of the same height. The first group answered 102 questions on average 
each week (towards a goal of 100), while those who found the tool most useful completed an additional 16 questions per week. Both groups had similarly low 
study times during the week and focused their efforts during the three hours before the weekly 10 pm deadline on Saturdays (marked by a vertical green line). 
The second group studied significantly more from 11 am to 7 pm on Saturdays, rather than adding in additional study periods on other days of the week. The 
self-review library offers students immediate, detailed feedback outside of business hours and the flexibility of being able to study at all hours of the day or night. 


a link to a tailored hint, and a link to the appropriate lecture 
slide from the parent module. Students can thus refresh the 
connection between the question and general topics, or re¬ 
ceive guidance on how to set up or think about the problem. 
After answering all five questions, students submit their work. 
They are presented with a solution set two seconds later, one 
which shows both the correct answers and the submitted an¬ 
swers for comparison. 

Students may choose to review a single module at a time or 
to study a sequence of modules. The selection process priori¬ 
tizes all modules within a sequence equally and all archetypes 
within a selected module equally, drawing from any archetype 
no more than once per quiz. All chosen modules thus have an 
equal probability of appearing on a quiz, and archetypes used 
to generate large sets of questions are not over-represented. 
Archetypes are not re-shown to a given student until all other 
archetypes within the parent module have been seen at least 
once, so that students encounter a variety of questions cov¬ 
ering all of the concepts within a module on their first pass 
through new materials. Questions within a single archetype 
are also not re-used until all versions have been used at least 
once, so that when archetypes re-appear they contain new spe¬ 


cific content. Students can thus review new concepts such as 
the relationship between the energy, wavelength, and color 
of an absorbed or emitted photon involved in an atomic tran¬ 
sition for as long as they wish without ever repeating exact 
questions. 

Students may also override the default selection process and 
enter challenge mode, where the archetypes which have been 
the most challenging for them initially are prioritized in pop¬ 
ulating quizzes. This is particularly useful to students once 
they think that they have mastered a new module, as they can 
then focus on the types of questions which gave them the most 
difficulty and prove themselves competent. Students typically 
study large sequences of modules for exams, and find the chal¬ 
lenge mode to be useful then for focused review of targeted 
topics. 

2.4. Self-Review Solutions 

Each solution set begins with a header noting the total score 
and a list of the questions in abbreviated form indicating 
quickly which ones were answered correctly. A complete so¬ 
lution is then presented, including a discussion of how deci¬ 
sions were made along the way. Equations are embedded into 
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TABLE 1 

Self-Review Library Question Distribution 


Module 


AT^ 

MC^ 

NV'* 

Total 

1. The Contents of the Universe 

80.2 


0.5 

66 

96 

31 

127 

2. Scientific Notation 

68.9 


0.6 

45 

112 

626 

738 

3. Timescales in the Universe 

70.9 


0.7 

53 

91 

47 

138 

4. The Phases of the Moon 

58.4 


0.7 

57 

597 

65 

662 

5. The Seasons on Earth 

54.9 


0.7 

53 

157 

164 

321 

6. The Origin of the Moon 

71.5 


0.7 

56 

81 

220 

301 

7. The Celestial Sphere 

67.9 


0.7 

59 

173 

99 

272 

8. Planetary Orbits 

65.9 


0.7 

46 

99 

47 

146 

9. The Scientific Method 

68.2 


0.7 

75 

732 

2,534 

3,266 

10. Geocentric and Heliocentric Models 

61.2 


0.8 

63 

116 

33 

149 

11. The Eormation of the Planets 

72.7 


0.7 

48 

83 

31 

114 

12. The Terrestrial Planets 

67.4 


0.8 

69 

85 

41 

126 

13. The Jovian Planets 

67.4 


0.8 

50 

84 

48 

132 

14. Waves and Light 

72.2 


0.7 

55 

1,499 

1.004 

2,503 

15. Atomic Structure 

62.5 


0.8 

46 

392 

40 

432 

16. Absorption and Emission 

64.7 


0.8 

60 

153 

55 

208 

17. Stellar Temperatures 

69.0 


0.8 

60 

150 

54 

204 

18. Nuclear Reactions 

57.3 


0.9 

53 

144 

47 

191 

19. Binary Stars 

60.0 


0.9 

51 

147 

220 

367 

20. The Hertzsprung-Russell Diagram 

65.9 


0.8 

48 

180 

860 

1,040 

21. White Dwarfs 

59.5 


0.8 

50 

99 

30 

129 

22. Neutron Stars 

66.2 


0.8 

48 

93 

48 

140 

23. Black Holes 

61.8 


0.8 

46 

81 

30 

111 

24. The Milky Way 

62.1 


0.9 

46 

200 

87 

287 

25. The Expansion of the Universe 

74.5 


0.8 

61 

80 

55 

135 

26. A Universe of Galaxies 

67.0 


1.3 

69 

111 

55 

166 

Average Number of Questions per Module 
Total Number of Questions 




55 

1,434 

224 

5,835 

253 

6,571 

477 

12,406 


^ ± cr: Mean and standard deviation of the mean for student answers 

^ AT: Archetypes of independent questions, from which entire families are formed 
^ MC: Multiple choice, ranking, or figure evaluation questions 
^ NV: Numerical value questions 


the discussion as small PNG-format images. In the case of 
mathematical questions, a fully worked solution is presented 
with the exact numerical values used in the particular prob¬ 
lem (not using variables to illustrate a general solution), so 
that students may check their work at every stage of the prob¬ 
lem. 

2 . 4 . 1 . An Instructor Immediacy and Affective Behavior Algorithm 

There is evidence that emotional state plays a role in learn¬ 
ing (cf., Craig et al. 2004). Studies have also shown that 
instructor immediacy, utilizing verbal and non-verbal com¬ 
munication techniques which act to reduce the perceived dis¬ 
tance (social and psychological) between instructor and stu¬ 
dent, has a positive effect on instructional outcomes and re¬ 
sults in increased cognitive and affective learning (Andersen, 
1979, pp. 543-559; Gorham, 1988; Witt, Wheeless, & Allen, 
2004). This type of analysis is being extended from evalu¬ 
ating in-class interactions to examine online interactions as 
well (Arbaugh, 2001; Rourke et al., 2001; Witt, 2004). An 
adaptive element, in which tutors provide targeted materials 
and feedback in response to individual student activity, is also 
acknowledged to be extremely effective (cf., Murray, 1999). 

The self-review library is designed to support student ef¬ 
forts to study privately yet interactively. Students, especially 
women, often mention this aspect positively - they appreci¬ 
ate that they can study a taxing concept for as long as it takes 


to learn it, without revealing, as they would if studying with 
peers, that it was difficult for them. However, because the 
system does grade the submitted material, users tend to re¬ 
late to it as a judge rather as a mentor or guide. Initial usage 
results suggest that students who view the library more as a 
personal guide and supporter and less as a taskmaster will en¬ 
gage more freely, and more often, with difficult topics, in ac¬ 
cordance with recent studies in affective learning (Burleson, 
2006). 

Human facilities studies (Fogg & Nash, 1997; Nass, 2004) 
have shown that humans frequently interact with computers 
automatically, applying and responding to social cues as they 
would with people, given even fairly minimal encouragement 
from a device that they know intellectually is not a social en¬ 
tity. People will be more likely to do a favor for a computer 
that has helped them (displaying reciprocity), and will treat 
a computer more kindly (with praise and greater attention) if 
it, or even another computer with a similar appearance, has 
recently provided them with helpful information. 

Two strong, basic cues of social support amongst humans 
are naming and empathy. A pilot algorithm has thus been 
launched examining how the interface can encourage stu¬ 
dents’ usage of the self-review library by addressing them by 
name and giving adaptive, positive feedback - rewarding cor¬ 
rect work and appropriate study patterns (such as spending 
appropriate amounts of time working on problems) - and by 
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offering sympathetic prompts in response to poor scores. 

Each solution set contains feedback provided by an adap¬ 
tive routine that evaluates the accuracy of the student’s an¬ 
swers, the amount of time spent per question, and the number 
of questions answered to date on a particular topic. The goal 
of this algorithm is to motivate students to succeed, encour¬ 
age them to improve their study habits by providing appro¬ 
priate feedback, and to present a more social persona for the 
self-review library through interaction. 

2.5. Programmatic Implementation 

The self-review library is implemented in dynamic HTML 
derived from an ASCII database on an Apache HTTP server. 
The HTML feature set is as conservative as possible, to allow 
for usage on both recent and outmoded versions of a range 
of web browsers. Every effort has been made to keep data 
file sizes to a minimum to increase accessibility. User names 
and access codes are stored in an encrypted format, and the 
back-end database controls have been written in Perl and C, 
optimized so that user requests take less than two seconds to 
fulfill. 

2.6. Student Portal and Interface 

Students cohorts enter the library through dedicated por¬ 
tals. They can select a single module or a range of modules 
to review, with default options updated every week to match 
their class schedule of topics. The article appendix contains 
examples of a quiz, a question hint, and a solution set. 

When students are confident that they have mastered certain 
modules through review, they can opt to take one of 16 weekly 
“formal” quizzes. Most weekly quizzes cover two modules at 
a time, and at the sequence half-point and end-point there are 
summary versions that cover either half or all of the mod¬ 
ules together. The key difference between review and weekly 
quizzes is that review quizzes (in effect, homework) contain 
links back into the supporting web pages and guided hints 
for each question, while weekly quizzes contain no such aids. 
Each weekly quiz is available for a full three-week window of 
time, to allow students to work ahead or behind by a week as 
necessary. 

At New Mexico State University (NMSU), homework 
scores are constructed from the review quizzes; students re¬ 
ceive the higher of (a) their average review quiz score, or (b) 
the average number of review questions completed per week 
(up to 100). A student who takes 20 five-question quizzes 
per week on average is thus guaranteed a homework score of 
100%, while one who takes 16 quizzes per week is guaranteed 
a homework score of 80% (and receives that score or their av¬ 
erage review quiz score if it is higher than 80%). Our policy 
encourages students to complete 100 questions per week, a 
goal achievable with several hours of study per week, which 
exposes them to a reasonable amount of content review and 
practice in problem-solving. The average score for the 16 
weekly quizzes is used to construct an independent course 
quiz grade, with no adjustment for effort. 

2.7. Progress Reports 

Students have access to progress reports which can be gen¬ 
erated on the fiy at any time. They show the number of 
quizzes completed and present average scores, including a 
breakdown into multiple choice and numerical value compo¬ 
nents. (Scores for numerical problems, which require students 
to perform a calculation and enter a numerical answer, are 


typically lower than those for other questions.) Two figures 
show the average scores and number of questions attempted 
by week and by lecture against clear target goals, to identify 
key modules for additional study. A third overview figure con¬ 
tains multiple indicators of progress, as shown in Eigure 2. A 
table of all weekly quiz scores is also provided so that students 
can monitor their progress and verify which weekly quizzes 
have been completed. 

2.8. Feedback Reports 

Students also have an opportunity to submit feedback after 
each quiz. This provides a mechanism for fiagging any er¬ 
rors in question implementation in real time, and also allows 
students to work directly with us if they feel that there are is¬ 
sues with questions or wish to simply give us their opinions. 
This tool is particularly helpful for students taking classes out¬ 
side NMSU (at other institutions), as it allows them to inter¬ 
act directly with the self-review developer (NPV) rather than 
having to funnel their feedback through their own instructors. 
Small extra credit “bounties” are offered for finding signifi¬ 
cant errors, but more importantly, we benefit from receiving 
student impressions of questions in real time - very helpful 
when adjusting scientific terms or figure points of view to be 
most accessible. 

2.9. Instructor Resources 

Instructors are provided with information for entire cohorts 
and for individual students. Summary HTML-format tables 
present average scores per module topic for each student (and 
group averages) and weekly quiz scores, as well as breaking 
down results into multiple choice and numerical value ques¬ 
tions, and evaluating review (homework) and the weekly quiz 
averages separately. 

Instructors can click on the name of any student from within 
the summary tables to review individual work. The inter¬ 
face will present for each student the materials within the stu¬ 
dent progress reports (described above), plus several enhance¬ 
ments. Tables break down the average scores and number of 
questions by module topic and by week, and sort most-missed 
question archetypes by lecture, ranked by student success so 
that the hardest questions for each student can be easily pulled 
up for review. 

Eigure 2 shows two instructor overview figures of student 
progress, also available for students to view. The first stu¬ 
dent record (top panel) represents successful learners, with a 
healthy mix of green and yellow boxes overall and few orange 
indicators of difficulty. Eilled-in columns represent study of 
topics-to-date for midterm and final exams; note that the col¬ 
ors for modules 4 and 13 (originally orange) shift to yellows 
and greens when reviewed before the midterm exam. The sec¬ 
ond student record (bottom panel) represents students having 
serious difficulty, as evidenced on first glance by the sea of 
red points. Boxes are smaller, indicating fewer study ques¬ 
tions have been attempted, and even when additional efforts 
are made (see modules 20 and 21) average scores do not rise 
significantly. The columns of red boxes during exam study 
periods indicate no additional success when reviewing mate¬ 
rials. Another common signature of difficulty (not shown on 
this record) is blank weeks where no work is done, followed 
by bouts of catch-up activity in which too many topics are 
covered at once for deep learning to occur. These figures can 
help instructors to identify potential problems of these types 
quickly, so as to intervene and provide additional guidance on 
productive studying habits. 
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Fig. 2.— Status indicators for two representative students, taken from progress reports which can be viewed at any time by students and their instructors. Each 
figure shows the number of questions answered within each lecture module over time, with the number of questions attempted each week scaling with the area 
of each box. Results are color-coded to indicate success rates above 80% (green), between 65% and 80% (yellow), between 50% and 65% (orange), and below 
50% (red). The top panel shows a record for a successful student, while the bottom panel shows the performance of a student having grave difficulties. 


Instructors are also provided with tables listing all quizzes 
and all quiz questions completed by each student, sorted by 
module or by date and divided into review mode, weekly 
mode, or showing all work. The date and time are shown 
for each quiz, as is the amount of time spent. Each entry links 
to a complete quiz recreation, showing student answers and 
correct answers and containing the original links to hints and 
lecture web pages. These quiz recreations can be very use¬ 
ful when meeting with a student one-on-one. They enable the 
instructor to discuss the amount of studying being done and 


the timing of the work, and allow instructors and students to 
review quizzes question by question and discuss appropriate 
strategies. They enable instructors to show explicitly how the 
information in a hint can be used to answer a question, or to 
identify patterns of mistakes shown in student answers (such 
as repeatedly multiplying rather than dividing in a particular 
type of units conversion). 

Instructor reports are compiled daily, and securely dis¬ 
tributed as encrypted, compressed archives that expand into 
a directory of HTML files conveniently navigated with any 
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Average Review Quiz Score 



Fig. 3.— Relationship between exam scores and efforts within the self-review library, comparing the results of tool usage to traditional indicators of performance 
for distance learners (top panels) and their in-class peers (bottom panels). The panels show the relationship of midterm and final exam scores to average review 
quiz (homework) scores at exam time. Linear fits are consistent with the data distribution, and correlation coefficients range between 60% and 80%. Note that 
the locus of points is shifted to higher exam and homework scores for the distance learners. 


web browser. 

2.10. Additional Self-Review Library Data Analysis Options 

A wealth of information is logged, allowing detailed analy¬ 
ses to be conducted on a variety of factors. Results are stored 
in an open format and can be analyzed with any tool (e.g., 
a spreadsheet or scripts). One can study the amount of time 
spent solving problems and reading solution sets, count the 
number of questions answered per topic or per unit time, plot 
the amount of pre-quiz practice done versus scores on weekly 
quizzes, compare scores for self-review and exam questions 
by topic, and track progress over time on a topic for indi¬ 
viduals or for a cohort. One can also “invert the analysis” 
and evaluate questions, comparing success rates for different 
questions (or sets) or for a question when answered at differ¬ 
ent times (e.g., on start, or after studying a particular topic for 
an hour). 

3. A COMPARISON OF USAGE OUTCOMES (REMOTE AND 
CLASSROOM COHORTS) 

It is reasonable to inquire how the use of online tutors af¬ 
fects learning. Previous studies (Aleven & Koedinger, 2002) 
have established a hierarchy of resources, ranking self-study 
as least effective followed by traditional classroom instruction 


0.75 standard deviations (cr) above (Corbett, 2001). Typical 
one-on-one human tutors rank 0.4cr higher (Cohen, Kulik & 
Kulik, 1982; Graesser, Person & Magliano, 1995), with intel¬ 
ligent computer-based tutors another 0.6cr higher (Anderson 
et al., 1995; Koedinger et al., 1997), topped by the best hu¬ 
man one-on-one tutors a full standard deviation above com¬ 
puterized systems based on meta-analyses of student perfor¬ 
mance (Bloom, 1984; Kulik & Kulik, 1991). There is also 
some evidence that computer-based systems allow students to 
master new material faster (Kulik & Kulik, 1991). With the 
use of such tools we seek not to replace human interaction, 
however, but rather to augment it, supporting tailored instruc¬ 
tion in situations where student-teacher ratios are highest and 
where students and teachers are isolated from each other and 
may work asynchronously. 

The self-review library utilizes several techniques deemed 
most efficient for learning. First, it employs distributed, rather 
than massed, practice of problem types, and constantly inter¬ 
leaves practice of multiple topics. Students typically study 
in multiple 20-minute sessions spread throughout each week 
of the semester, and each review session is focused around 
two lecture modules containing 100 question archetypes and 
covering six key concepts for the week. Rather than read¬ 
ing full textbook chapters and then switching over to solving 
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problems in blocks, our materials are interwoven so that on 
a question-by-question basis students can consider a single 
problem, review a single lecture slide, listen to supporting au¬ 
dio describing applications of new concepts, and process a 
hint that guides them in how to set-up the problem success¬ 
fully. This helps students to read new material in an informed 
manner, constantly assessing what can be derived from it to 
apply to a range of relevant questions rather than absorbing 
the factual content at face value (without a sense of valid 
application). Students must assess each question and deter¬ 
mine an optimal strategy to solve it, forming linkages between 
classes of questions and matching strategies, rather than repet¬ 
itively applying a single rule to a large group of problems of 
a single type. Though this typically requires more effort and 
doesn’t deliver the rapid improvement associated with massed 
practice of a single problem-solving strategy, it leads to bet¬ 
ter mastery and a increased retention of concepts for a longer 
time (Roediger, 2013). Recent studies (cf., Rohrer, Deder- 
ick, & Burgess, 2014) have found that interleaved rather than 
blocked practice can be almost twice as effective, highlighting 
the utility of this technique. 

Second, the library stimulates retrieval practice (Bjork, 
1975; Roediger & Ka^icke 2006; Soderstrom & Bjork, 2013) 
in its users through its basic format, a semester-long series 
of short review quizzes that constantly drive students to draw 
knowledge from memory and solve a variety of questions. Ef¬ 
fective retrieval practice occurs when students employ it over 
a series of staggered sessions (rather than in single bursts of 
mindless repetition; Roediger, 2013). Our students may ac¬ 
cess a variety of aids to solve each question, as detailed above, 
but the knowledge that the formal weekly quizzes and the 
course exams will be conducted without these aids provides 
motivation for them to use aids initially, but over time strive 
to answer questions by drawing on memories instead, thus 
consolidating those memories. Because students can achieve 
100% scores for their review (homework) activities simply by 
meeting the target goal for number of problems solved (100 
per week), these short quizzes can be treated as learning op¬ 
portunities rather than a testing environment with pressure to 
meet a set standard score (particularly helpful for classes with 
a broad range of previous mathematical and scientific expo¬ 
sure). 

A second question of merit is how online tutors can help us 
to provide traditional science coursework to distance learners, 
to expand the reach of higher education and improve retention 
for dispersed populations. In a separate article (Vogt, Cook, & 
Muise, 2013) we address one major challenge for the physical 
sciences, namely providing valid experimental laboratory ex¬ 
periences to students working remotely with no access to spe¬ 
cialized laboratory equipment and training. We now present 
evidence that our distance learners are performing at least as 
well as parallel in-class cohorts of students on lecture materi¬ 
als, with the aid of our GEAS online tutor. 

Eigure 3 shows the correlation between study within the 
self-review library and traditional exam performance for a re¬ 
mote cohort of distance learners and an in-class set of peers at 
NMSU. Exam scores are plotted against average review quiz 
(homework) scores at the time of each exam, showing a linear 
relationship in each case. 

Eigure 4 focuses on performance within the library for 
these two cohorts and another in-class group of undergradu¬ 
ates from Humboldt State University (HSU). We note that the 
three cohorts exhibit the same relative peaks and dips in aver¬ 
age quiz score for various lectures, with the distance learners 


offset by -i-5% relative to the in-class groups on average. They 
also complete 20% more questions (averaging around 65 per 
lecture, with a nominal target of 50 for all groups), though 
note that the bulk of their extra studying takes place over the 
first half of the semester. 

In Eigure 5 we divide the distance learners into two groups 
by gender, and observe that within this cohort men outscore 
women by 7 ± 0.6% on average across the board. We do not, 
however, observe this offset in either of the in-class cohorts 
under study. 

The study of gender disparities in the physical sciences is a 
complex field unto itself. In a recent meta-analysis of 26 stud¬ 
ies of concept inventories in physics, Madsen, McKagan, and 
Sayre (2013) grappled with the effects of 30 factors, including 
teaching and engagement techniques, student background and 
preparation, and stereotype bias. They found that no single 
factor could explain the differences in outcomes with gender. 
While the complete underlying causes of gendered effects are 
beyond the statistics available to us for our data set we can 
explain the observed difference between distance learners and 
classroom students, and suggest a possible cause for it as well. 

Like Madsen, McKagan, and Sayre, we find an offset be¬ 
tween men’s and women’s scores on a pre-class concept in¬ 
ventory. The difference is 16 ± 3% for the distant cohort, 
but only half as much (7.9 ±3%) for the classroom students. 
The amount of time the genders spend studying within the 
self-review also differs between cohorts. Eor distant learners, 
men spend 5 minutes less per week answering questions (56 
versus 61 minutes) but answer the same number of questions 
(65 per week, with a nominal goal of 50 per week). Eor the 
classroom students, however, men spend 11 minutes less per 
week answering questions (40 versus 51 minutes) and also 
answer significantly fewer questions (44 versus 62 per week). 
This suggests that weaker study habits of the male cohort in 
the classroom cohorts throughout the semester act to diminish 
the performance difference between genders. 

We speculate that one reason for this difference in observed 
study habits is a selection bias in course population. Our 
in-class cohorts contain 75% freshman and sophomores and 
25% juniors and seniors, while these fractions are completely 
reversed in the distant cohort (which also tends to be a few 
years older). The more senior students in the distant cohort 
have less fiexibility in fulfilling their general educational sci¬ 
ence coursework requirement (our astronomy course is cur¬ 
rently the only qualifying science course at NMSU which can 
be completed remotely, while there are tens of options on- 
campus within astronomy, biology, chemistry, geology, and 
physics), and their clock are ticking on time to degree com¬ 
pletion. They thus prioritize our course by necessity, as the 
cost of failure is higher. 

We can also address the issue of how student performance 
improves over time spent studying within the self-review li¬ 
brary (see Eigure 6). We analyzed the average score on ques¬ 
tions over time within each lecture for each student in vari¬ 
ous cohorts, sorting questions in the order in which they were 
attempted and aligning the first question on each lecture for 
each student together, followed by the second question on 
each lecture for each student, and so on. These 1,600 curves 
of growth (for 26 lectures from 61 students) were then aver¬ 
aged together to create averaged curves showing the change 
in accuracy over time. Curves were truncated at the point at 
which only 30 individual samples remained to be averaged 
together, and also smoothed with a simple boxcar filter to re¬ 
move the small-scale variations and highlight global behav- 
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Fig. 4.— Distribution of average scores (with 3 sigma error bars on each mean value) and number of questions attempted within the self-review library as 
a function of lecture module, for three student cohorts (distance learners in solid green, and two in-class cohorts in dashed blue and dotted red). The left axis 
labels average scores (line plots above), and the right axis labels the number of questions (histograms below). The horizontal yellow line represents the nominal 
goal of attempting 50 questions per lecture module and 1600 questions per semester. The peaks in number of questions attempted appear at the beginning of the 
course and the first lecture of the second half (after the midterm exam), while the low for the final lecture module reflects students shifting from weekly module 
study into second half review for the final exam. The pattern of high and low scores is reproduced in both cohorts, and is caused by variations in the difficulty 
of material from lecture to lecture. The distance learners attempt 10 more questions per lecture on average than the in-class cohorts; all groups hover around the 
nominal goal of 50 questions per lecture. They also score 5% higher on average, in part due to completing more questions per lecture. 



Fig. 5.— Distribution of average scores and number of questions attempted within the self-review library as a function of lecture module, for distance learners. 
Figure format is otherwise as in Figure 4. Data are separated into results for men (blue solid lines) and women (red dotted lines). As in Figure 4, the pattern 
of high and low scores is reproduced in both cohorts, and is caused by variations in the difficulty of material from lecture to lecture. Male and female students 
complete similar numbers of questions (though women spend 8% more time on them, for 61 minutes per week), but men score 7% higher on average. 





























































10 


An Online Tutor for Astronomy: The GEAS Self-Review Library 


iors. 

Our first, over-riding point is that the curves all rise. Over 
time, students learn new concepts and absorb new techniques 
for solving problems, and are able to apply them successfully. 
We have already shown that average scores within the library 
correlate well with traditional measures of performance (see 
Figure 3), and we can also observe the systematic acquisition 
of knowledge and skills over time within the learning envi¬ 
ronment. 

At first glance, one may notice that initial scores may be¬ 
gin as low as 55% for some cohorts. This is due in part to 
the fact that these classes are for non-science majors, and 
host many students whose last contact with mathematics is a 
long-distant, faded memory. More important, however, is the 
way that we encourage students to structure their studying. 
Many of our students cling to traditional study habits (e.g., 
re-reading textbooks and underlining or highlighting key sec¬ 
tions) even though the current understanding (Roediger, 2013) 
suggests strongly that these techniques are neither efficient 
nor beneficial to learning or performance. We want our stu¬ 
dents to learn by doing (by combining diverse facts to draw 
conclusions, and by solving problems), and to utilize meth¬ 
ods such as retrieval practice, a focus on interleaved topics, 
and pre-testing on new material to optimize their learning (cf.. 
Little & Bjork, 201 1) as we best understand how to do so. We 
thus encourage them to begin their study of a new lecture by 
initially requesting a review quiz (beginning their homework) 
as a first step. Their initial scores can be quite low because of 
this, as they acquire information question by question, but the 
advantage is that they immediately see the applications and 
the purpose of facts under study. 

In the left panel of Figure 6 we present three curves for 
the cohorts shown in Figures 3-4, with distance learners in 
green and two in-class cohorts in red and blue (with colors 
as in Figure 4). For all three cohorts of students there is a 
clear increase in performance with time, and studying contin¬ 
ues well past the nominal goal of 50 questions per lecture for 
each student. The distance learners complete more questions 
per lecture on average. Their average scores continue to rise 
with the additional exposure to material, but are also higher 
on average when they are compared to the in-class students at 
the same point along the curves (having answered the same 
number of questions on average). 

The middle panel of Figure 6 reproduces the same green 
curve showing the cohort of distance learners, and then splits 
this group into those with final grades in the top (blue) and 
bottom (red) halves of the group. We observe that all students 
improve performance over time, but those who perform best 
in the course begin at a higher level and also improve at a 
faster rate over the time interval beyond the 50-question limit 
(and continue to study additional questions, productively). 

TABLE 2 

Comparison of On-Campus and Remote Astronomy Cohorts 


Cohort 

Exam Scores 

Enrollment 

Withdrawals 

Classroom 

76.6 ± 1.5 

85 

14 

Remote 

79.6 ± 1.7 

97 

36 


identified a group of “super-studiers”, composed of 29% of 
the students, those who completed between 340 and 754 
quizzes over a semester (between 20 and 434, or 6% and 
136%, beyond the nominal goal of 320). Because our grading 
algorithm guarantees students a score of 100% for homework 
as long as students complete 20 quizzes per week (320 per 
semester), there is no direct increase in scores for any review 
work done beyond this limit. The students who choose to 
complete more review work thus appear to be doing so be¬ 
cause they find it to be a beneficial strategy for studying. The 
two groups show identical improvement in accuracy up to the 
nominal 50-question goal line. Beyond this line, however, we 
observe that the normal group continue to study but shows 
no strong increase in accuracy, while the super-studiers an¬ 
swer twice as many questions beyond the nominal goal and 
continue to increase in accuracy over this interval. These stu¬ 
dents are not necessarily the strongest learners in the class nor 
those with the best backgrounds in math and science upon en¬ 
tering the course. They simply find this mode of study to work 
exceptionally well for them. 

Two versions of the NMSU general astronomy course were 
taught by the same instructor, one on-campus during Fall 2010 
and one via distance learning from Fall 2011 - Spring 2014. 
Written midterms and finals are never returned, so 88% of the 
two-hour final exam was identical across both cohorts (see 
Table 2 and Figure 3). Distance learners performed slightly 
better, with a mean value two pooled standard deviations of 
the mean above that for the in-class students. (A two-sample 
t-test yields an inconclusive less than 9% probability that the 
two scores were drawn from the same parent sample.) Our 
results are promising, and suggest that our distance learners 
perform at least as well as our in-class students. 

Note that the distance learning cohort had a larger frac¬ 
tional rate of withdrawal (twice that of the in-class course), 
as expected (Rovai, 2003; Woodley, 2004). All students per¬ 
formed similarly on a pre-class astronomy concept inventory 
(Halloun & Hestenes, 1985), and reported equal experience 
with computers. Those who completed the distance learning 
course were more likely to have studied math beyond algebra 
(though their self-assessed math skills were no higher). How¬ 
ever, half of the students who withdrew did so without com¬ 
pleting any course work, mainly due to difficulties securing 
financial aid. Other common reasons cited were difficulties 
spending enough time on the course, transfer to an in-class 
course, and a birth or death in the family. This is consistent 
with findings in the literature that non-academic factors dom¬ 
inate academic factors in driving withdrawal rates in distance 
learning (cf., Subotzky & Prinsloo, 2011). 

4. EXAMPLES OF SELF-REVIEW LIBRARY USAGE MODES 

There are four primary modes in which the self-review li¬ 
brary is designed to be used. 

4.1. Semester-Long Class Sequence 

The first mode applies to a class of students working 
through a full semester timeline. Students use the review and 
the weekly quiz tools and study all 26 lecture modules. This 
model works well for early-career faculty who have not yet 
invested significant time in developing their own lecture se¬ 
quences, who opt to teach entirely from our materials. 


The right panel of Figure 6 shows the same cohort of dis¬ 
tance learners, again split into two groups. In this case we 
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Fig. 6.— Improvement in scores for review quiz questions (homework) over time, as more questions are answered within a given lecture module. Students 
start out with average accuracy levels around 55%, increasing to 70% or 80% as they master new concepts and techniques for problem solving. In the left panel 
the green line of starred triangles indicates the progress for distance learners, while the blue solid dots and red open squares show results for two in-class cohorts. 
The distance learners start out at a slightly higher level of accuracy, and are performing significantly better by the 50-question mark (the nominal goal per lecture 
module, shown as a vertical line). For those students who continue to study, additional exposure results in significantly higher scores over time. Distance learners 
answer more questions on average, and end up with higher scores. In the middle panel, the cohort of distance learners (green line of starred triangles again) starts 
out with accuracy levels around 55%, increasing to almost 80% on average as students master new concepts and techniques for problem solving. The cohort 
is then divided into two equal-sized groups based on final course grade. The high-grade group (blue dots) begins their studies with scores 12% higher than the 
low-grade group (red open squares) and also makes considerably more progress over time, with scores 15% higher at the 50-question mark and 20% higher when 
the low-grade group stops studying, increasing by another 3% as they complete an additional 40 questions on average. Students who perform better across the 
board in the class answer more questions on average, and end up with higher scores. In the right panel the distance learning cohort is redivided into students who 
complete the nominal goal of 320 quizzes (1600 questions) over a semester (red open squares) and those who complete more than 340 quizzes (blue solid dots). 
The two curves match through the recommended 50-question period, but after this point the normal group plateaus in score and halts studying, while those who 
persevere longer more than double their improvement in score. 


4.2. General Study Tool 

The second mode applies to a class of students working on 
an independent timeline, covering many but not all of the 26 
GEAS modules. These students do not take formal weekly 
quizzes. They select sequences of modules to study based on 
their instructor’s preferences (presented to them as default op¬ 
tions at the correct times through a dedicated portal interface), 
and utilize the library as a personal study tool and to gain ex¬ 
tra experience in working problems. This model works well 
for faculty with existing teaching materials of their own who 
wish to provide addition problem-solving experience to their 
students and/or to satisfy state or college requirements to pro¬ 
vide external activities for students. 


work and interface controls of the library are cloned, and a 
parallel, independent archive is used to populate a self-review 
library for another field of study. 

Collaborators are currently being solicited in all four 
modes, and inquiries are welcome from instructors interested 
in working with our materials in the classroom and for dis¬ 
tance education cohorts. Individual test accounts in the self¬ 
review library are also available by request. More information 
on the General Education Astronomy Source (GEAS) may 
be found at http://astronomy.nmsu.edu/geas, or by contacting 
geas @ astronomy.nmsu.edu. 


4.3. Targeted Activity 

The third mode applies to groups conducting a focused 
study of particular topics within the self-review library. These 
individuals work within a “closed box” subset of the library. 
All quizzes are conducted in review mode, and the progress 
reports and feedback forms are modified to refiect the short¬ 
term nature of the exercise. This model works well for studies 
evaluating new learning tutorials or aids. 

4.4. Infrastructure Adaptation 

The fourth mode applies to instructors and developers inter¬ 
ested in utilizing the framework of the self-review library to 
develop online tutorial offerings for other topics. The frame¬ 


This material is based in part upon work supported by the 
NSE through Grant No. AST-0349155 to NPV and by NASA 
through Grant No. NNX09AV36G to NPV. Any opinions, 
findings and conclusions or recommendations expressed in 
this material are those of the authors and do not necessarily 
refiect the views of the NSE or NASA. 

We are pleased to thank the students and instructors of New 
Mexico State University and Humboldt State University who 
participated in our pilot program. 

5. APPENDIX 

Three attached figures show a sample self-review quiz (Eig- 
ure 7), created in dynamic HTML by request, hint (Eigure 8), 
and solution set (Eigure 9). All such materials are archived 
within the class records for easy access by the instructor. 
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#1. The plot drawn below shows the decay of a radioactive sample of a certain element with time. Which 
of the following statements about it are true? 



The amount of the original element which remains in the sample, as a function of time. 

0[1] The half-life of this element is between 5,000 and 7,000 years. 

0[2] Twice as many atoms decay during the first half-life than during the second shown. 
0[1] and [2] are both true. 

0[1] and [2] are both false. 



#2. If a star has a temperature of 6810 K (recall that the temperature of the Sun is 5800 K), and is 109 
times as large as the Sun, how bright is it? The luminosity, in solar units, is: 


Write your answer in the box, using a standard format for numbers. For example, if the luminosity is one hundred 
and twenty five times brighter than that of the Sun type "125" or "1.25e2". If the luminosity is eight times fainter 
than that of the Sun type "0.125" or "1.25e-1". 


Fig. 7.— Sample questions from a self-review quiz, containing links to hints for solving the question (“?” button), and a link back to the most relevant lecture 
slide (“i” button). Both a representative multiple choice and numerical value question are shown. Radioactive decay is introduced as an age-dating mechanism 
as we discuss the formation of the solar system in Module 13, and the evolution of stars within the parameter space defined by luminosity (energy output), 
temperature, and size is the focus of Module 20. 
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A radioactive half-life is the amount of time it takes for half of the atoms in a sample of a given radioactive element to decay, 
changing from an unstable form into another state. During the initial half-life, twice as many atoms will decay as during the 
second half-life. During the second-half life twice as many atoms will decay as during the third half-life, etc. 

When determining the half-life of a radioactive sample based on a plot of its decay rate, divide the total count rate (the 
number of atoms) for that sample by two. Draw a horizontal line at that value on the y-axis (the vertical axis). At the point 
where your line meets the plotted curve, draw a vertical line down to the x-axis (the horizontal axis). The value at which your 
vertical line meets the x-axis will tell you the amount of time that elapsed during a single half-life. 

In the figure shown below, we begin with a total number of 5(X) million atoms. We draw a horizontal line at a height of 250 
million, extend it over to the right, and observe that the sample decreases from 500 to 250 million radioactive atoms over a 
time period of 20 years. Repeating the process for a second half-life, we confirm that in another 20 years the sample 
decreases from 250 to 125 million radioactive atoms. 



0 10 20 30 40 

Time (years) 


Keep in mind that the atoms which decay do not disappear into thin air - they are merely converted into another element or 
isotope. 


Fig. 8.— The hint for question #1 shown in Figure 7, accessed by clicking on the “?” button within the question. Most students make frequent use of the hint 
option while studying, as well as the parallel link to the relevant lecture slide for each question. 
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#1. The plot drawn below shows the decay of a radioactive sample of a certain element with time. Which 
of the following statements about it are true? 



The amount of the original element which remains in the sample, as a function of time. 

0[1] The half-life of this element is between 5,000 and 7,000 years. 

0[2] Twice as many atoms decay during the first half-life than during the second shown. 

©[1] and [2] are both true. 

0[1] and [2] are both false. 

Our sample begins with 100 counts, and we know that over the course of each half-life 50% of the remaining 
atoms of the radioactive element will decay. Over the course of one half-life, the first 100 counts drop to 50 
counts. Measuring on the plot, we see that the time (measured along the x-axis) over which the amount of the 
original sample (measured along the y-axis) drops from a level of 100 to 50 is just less than 6,000 years. 

We know that in each half-life, 50% of the remaining atoms of the radioactive element will decay. Over the first 
half-life, we begin with 100 counts and decay to 50 (losing 50). Over the second, we begin with 50 counts and 
decay to 25 (losing 25). Twice as many atoms do indeed decay during the first half-life as the second, because 
the sample is so much larger when it starts. 


#2. If a star has a temperature of 6810 K (recall that the temperature of the Sun is 5800 K), and is 109 
times as large as the Sun, how bright is it? The luminosity, in solar units, is: 

22600 (2.55e19 is incorrect) 

We use Stefan's Law to relate the luminosity L, temperature T, and radius R of the star. Because we know the 
values of R and T, we will express the relation in terms of L as a function of R and T. 


L = R^xT^ 


The radius R is 109 times that of the Sun. The temperature T is 6810 K. We need to rewrite T in terms of the 
temperature of the Sun (which has a temperature of 5800 K). 


T = mi0K = (S)t® = 1.174 7® 


We now insert the values of R and T into the equation to find the value of L. 


L = 109- X 1.174^ = 22,600 L© 


To check our numerical answer, we estimate the location of the star on the Hertzsprung-Russell diagram. At the 
intersection of a temperature of 6810 on the x-axis and the diagonal line which would represent a radius of 109, 
we draw a horizontal line over to the y-axis to find that stars located here should have a luminosity roughly 
equivalent to that which we calculated. Our answer has been verified! 


Fig. 9.— A portion of a solution set, for the two quiz questions shown in Figure 7. A worked solution is presented for each problem, noting both student 
answers and correct ones. The figure in question #1 has been annotated to help students to understand it; figure reading is a new skill for many of our students. 
In question #2, the student neglected to convert the stellar temperature from kelvins into solar units, resulting in an absurdly large luminosity. The value needed 
to lie within 250 Lq of 22,600 Lq in order to be accepted as correct, giving a reasonable amount of room for rounding errors in the calculation. 

















